Robotic cooperative system

ABSTRACT

An automatic method for autonomous interactions between robots, comprising an action of automatically receiving, by a transport robot, a request for transporting a service robot. The method comprises an action of automatically computing a location of the service robot. The method comprises an action of automatically moving the transport robot to the location of the service robot. The method comprises an action of automatically sending a signal from the service robot to the transport robot using a signal emitter incorporated into a mechanical element attached to the service robot. The method comprises an action of automatically coupling, using the signal, the mechanical element to a carrier element attached to the transport robot.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority from U.S. Patent Application No.62/247,212, filed Oct. 28, 2015, entitled “A robotic system formaintaining, servicing, and monitoring surfaces and solar energypanels”, the contents of which are incorporated herein by reference intheir entirety.

FIELD OF THE INVENTION

The invention relates to the field of robotics.

BACKGROUND

A robot may be described as an electromechanical machine that may beguided by a computer program or electronic circuitry, such as anembedded system and the like. Robots may be autonomous, semi-autonomous,or operator controlled, and may perform one or more tasks, such asindustrial robots, medical operating robots, patent assist robots, dogtherapy robots, collectively programmed swarm robots, unmanned aerialvehicle (UAV) drones, unmanned ground vehicle (UGV) drones, unmannedmarine vehicle (UMV) drones, microscopic nano-robots, and the like. Arobot may have electronic circuitry or control systems that provideinstructions for the operation and/or motion of the robot whenperforming the one or more task.

Robotics technology deals with the design, construction, operation, andapplication of robots, as well as computer systems for their control,sensory feedback, and information processing. These technologies dealwith automated machines that may perform one or more tasks, such astasks in dangerous environments, manufacturing processes, repetitivetasks, dangerous tasks, and/or the like.

Navigation robots have been used to transport functional robots betweensites where the functional robots perform the one or more tasks. Thesenavigational and functional robots may be combined into a robotic systemfor managing multiple functional robots and transporting them betweenthe sites where tasks may be performed.

The foregoing examples of the related art and limitations relatedtherewith are intended to be illustrative and not exclusive. Otherlimitations of the related art will become apparent to those of skill inthe art upon a reading of the specification and a study of the figures.

SUMMARY

The following embodiments and aspects thereof are described andillustrated in conjunction with systems, tools and methods which aremeant to be exemplary and illustrative, not limiting in scope.

There is provided, in accordance with an embodiment, a method forautonomous interactions between robots, comprising an action ofreceiving automatically, by a transport robot, a request fortransporting a service robot. The method comprises an action ofcomputing automatically a location of the service robot. The methodcomprises an action of moving automatically the transport robot to thelocation of the service robot. The method comprises an action of sendingautomatically a signal from the service robot to the transport robotusing a signal emitter incorporated into a mechanical element attachedto the service robot. The method comprises an action of couplingautomatically, using the signal, the mechanical element to a carrierelement attached to the transport robot.

In some embodiments, the method further comprises an action oftransporting automatically the service robot by the transport robot to anew location, and an action of releasing automatically of the mechanicalelement from the carrier element by the transport robot.

In some embodiments, the new location is determined by: an action oflocating automatically a region of the new location; an action of movingautomatically of the transport robot with the coupled service robot tothe region; an action of receiving automatically a sensor signal forlocating the new location within the region; and an action of movingautomatically of the transport robot to the location using the sensorsignal.

In some embodiments, one or more of the location and the new locationare work surfaces.

In some embodiments, the work surfaces are one from a group consistingof solar panels, ship hulls, airplanes hulls, vehicle hulls, buildingwindows, concentrated solar power plant mirrors, and turbine blades.

In some embodiments, the method further comprises an action of orientingautomatically the service robot parallel to one or more of the worksurfaces to be serviced before the releasing, wherein the mechanicalelement comprises at least two parts, and wherein the orientingautomatically is performed by articulation of the at least two parts.

In some embodiments, the articulating automatically is one or more fromthe group consisting of (i) an active articulation guided in real-timeby controller instructions, (ii) fixed articulation for a preconfiguredsurface angle, and (iii) a passive articulation preconfigured for arange of surface angles, wherein the passive articulation comprises anarticulating joint comprising a fixed angle element and a flexible angleelement.

In some embodiments, the method further comprises an action of thetransport robot servicing automatically one or more of the worksurfaces.

In some embodiments, the method further comprises locating automaticallya region of the service robot by the transport robot, and an action ofmoving automatically of the transport robot to the region of the servicerobot.

In some embodiments, the mechanical element is attached to the transportrobot and the carrier element is attached to the service robot.

In some embodiments, the signal comprises an identification pattern.

In some embodiments, the method further comprises an action oftransferring automatically of one or more of electrical signals, power,and servicing materials between the transport robot and the servicerobot.

In some embodiments, the method of claim 1, further comprising receivingautomatically a drop-off request for the service robot.

In some embodiments, the drop-off request comprises one or more of alocation and a region.

There is provided, in accordance with an embodiment, a robot couplingsystem comprising one or more service robot comprising a mechanicalelement, wherein the mechanical element comprises a localization signalemitter. The robot coupling system comprises one or more transport robotconfigured to receive automatically a request for transporting the oneor more service robot. The one or more transport robot comprises acarrier element configured to couple automatically with the mechanicalelement. The one or more transport robot comprises a sensor configuredto receive automatically a localization signal from the localizationsignal emitter. The one or more transport robot comprises a motorconfigured to move automatically the one or more transport robot from alocation to a new location. The one or more transport robot comprisesone or more hardware processor. The one or more transport robotcomprises one or more storage unit comprising processor instructionencoded thereon.

In some embodiments, the one or more storage unit comprises processorinstruction for instructing the one or more hardware processor toreceive automatically a request for transporting the one or more servicerobot. The processor instructions instruct the one or more hardwareprocessor to compute automatically a location of the one or more servicerobot and move the one or more transport robot to the location of theone or more service robot. The processor instructions instruct the oneor more hardware processor to receive automatically a signal from theone or more service robot to the one or more transport robot using theemitter. The processor instructions instruct the one or more hardwareprocessor to mechanically couple automatically the mechanical element tothe carrier element using the signal. The processor instructionsinstruct the one or more hardware processor to transport automaticallyof the one or more service robot by the one or more transport robot to anew location. The processor instructions instruct the one or morehardware processor to release automatically of the mechanical elementfrom the carrier element by the one or more transport robot.

In some embodiments, the robot coupling system further comprisesprocessor instructions instruct the one or more hardware processor tolocate automatically a region of the one or more service robot by theone or more transport robot, and move automatically of the one or moretransport robot to the region of the one or more service robot.

In some embodiments, the new location is determined by: locatingautomatically a region of the new location; moving automatically of theone or more transport robot with the coupled one or more service robotto the region; receiving automatically a sensor signal for locating thenew location within the region; and moving automatically of the one ormore transport robot to the location using the sensor signal.

In some embodiments, one or more of the location and the new locationare work surfaces.

In some embodiments, the work surfaces are one from a group consistingof solar panels, ship hulls, airplanes hulls, vehicle hulls, buildingwindows, concentrated solar power plant mirrors, and turbine blades.

In some embodiments, the robot coupling system further comprisesprocessor instructions for orienting automatically the service robotparallel to one or more of the work surfaces to be serviced before thereleasing.

In some embodiments, the mechanical element is attached to the transportrobot and the carrier element is attached to the service robot.

In some embodiments, the localization signal comprises andidentification pattern.

In some embodiments, the mechanical element and carrier element areadapted to transfer of one or more of electrical signals, power, andservicing materials between the transport robot and the service robot.

There is provided, in accordance with an embodiment, a robot couplingassembly comprising a mechanical element comprising a localizationsignal emitter, wherein the mechanical element is attached to a servicerobot. The robot coupling assembly comprises a carrier elementconfigured to couple with the mechanical element, wherein carrierelement is attached to a transport robot.

In some embodiments, the mechanical element comprises a shaft and a malesecuring element, wherein the male securing element is larger, in atleast one dimension, than the shaft.

In some embodiments, the carrier element comprises a guidingsub-assembly and a female securing element, wherein the female securingelement is substantially the at least one dimension of the shaft suchthat the male securing element is coupled securely within the femalesecuring element.

In some embodiments, the guiding sub-assembly comprises an opening andtwo guiding elements, wherein the opening is distal from the femalesecuring element, wherein the at least one dimension of the opening issubstantially larger than the at least one dimension of the shaft, andsuch that the guiding elements form a funnel between the opening and thefemale securing element.

In some embodiments, the two guiding elements are elongated materialelements with two or more bends to define the opening and the funnel forguiding the mechanical element into the carrier element.

There is provided, in accordance with an embodiment, a robot systemcomprising

two or more service robots, each configured to perform automatically atleast on servicing task on an object, and at plurality of transportrobots, each configured to carry automatically one or more of theplurality of service robots between multiple sites, each site comprisingone from the group consisting of one or more object, a mobile groundstation, a charging station, a repair station, and a base station.

In addition to the exemplary aspects and embodiments described above,further aspects and embodiments will become apparent by reference to thefigures and by study of the following detailed description.

BRIEF DESCRIPTION OF THE FIGURES

Exemplary embodiments are illustrated in referenced figures. Dimensionsof components and features shown in the figures are generally chosen forconvenience and clarity of presentation and are not necessarily shown toscale. The figures are listed below.

FIG. 1 shows a schematic illustration of a system for coupling between atransport robot and a service robot;

FIG. 2 shows a flowchart of a method for coupling between a transportrobot and a service robot;

FIG. 3 shows a schematic illustration of a carrier element for couplingbetween a transport robot and a service robot;

FIG. 4 shows a schematic illustration of a mechanical element forcoupling between a transport robot and a service robot;

FIG. 5 shows a schematic illustration of a coupling between a transportrobot's carrier element and a service robot's mechanical element;

FIG. 6 shows a schematic illustration of an isometric view of a servicerobot coupled to a transport robot's carrier element using the servicerobot's mechanical element; and

FIG. 7 shows a schematic illustration of a front view of a service robotcoupled to a transport robot's carrier element using the service robot'smechanical element.

DETAILED DESCRIPTION

Described herein is a technique for using two or more types of robots,one of which is a transport robot, to service fields containing multipleseparated surfaces. The transport robots may automatically move theother types of robots, such as service robots, between surfaces toservice them. Described are embodiments of methods and devices for theautonomous operational and mechanical interactions between the two typesof robots. The operational interactions may use mechanical couplingelements attached to each robot, such as an arm mechanical element, acarrier mechanical element, and/or the like, where coupling between themechanical elements provides a transient mechanical connection betweenthe transport and the service robots. The mechanical coupling elementsmay be configured to have at least dimension on one element to be largerthan the corresponding dimension on the other element, so that theelements may couple together to support the weight of one robot by theother. Optionally, at least two dimensions of one mechanical elementsare larger than the corresponding dimensions of the other mechanicalelement. The mechanical coupling elements may also incorporate matchedemitters and sensors that exchange localization signals, which improvethe coupling be sending motion signals to the respectively attachedrobots which allow the relative positioning between the two robots sothat the coupling occurs without incident. For example, the emitter is alight source and the sensor is a camera. For example, the emitter is aradio wave directional antenna and the sensor is a directional receivingantenna. For example, the emitter is an acoustic wave generator and thesensor is an acoustic sensor.

Optionally, the localization signals sent automatically by an emittercomprise a signaling pattern that is unique to each individual robot,thereby improving the ability of a transport robot to locate anindividual service robot without a crowd of service robots using theunique signaling pattern.

Optionally, electrical power, electrical signals, servicing materials,and the like are transferred automatically between a service robot andthe transport robot. For example, measured values collected by sensorson a service robot are transferred automatically to the transport robot,and vice versa. For example, values from camera sensors on a servicerobot are transferred automatically, such as wirelessly, though thecoupling connection, and the like, for identifying automatically a newlocation to detach the service robot from the transport robot.Optionally, signals, materials and/or energy is transferredautomatically between a service robot and a transport robot. Forexample, the coupling elements include electrical and material transportconnections that are used to automatically transfer energy and/ormaterials, such as cleaning fluids.

For example, a digital communication is transferred automatically fromthe service robot to the transport robot, such as a message from theservice robot that it is safe for the transport robot to release theservice robot. For example, a cleaning fluid is transferred between thetransport robot and the service robot. For example, the service robottransfers electrical power/energy from the service robot's battery tothe transport robot's battery. These automatic transfers may allowsynergistic cooperation between the robots before, during, or aftercoupling to allow safe pickup/transport/drop-off. These transfers mayallow automatic system-wide optimization of resources, such as energy,materials, and the like.

Optionally, sensors on a transport robot are connected to the transportrobot such that a coupled service robot may not interfere with readingsfrom the sensors, such as sensors attached to elongated mechanicalappendages of the transport robot. For example, sensor values collectedby the service robot are transferred automatically to the transportrobot.

Reference is now made to FIG. 1, which is a schematic illustration of asystem 100 for coupling between a transport robot and a service robot.System 100 comprises a service robot 101 and a transport robot 102.Service robot 101 services one or more work surfaces 110. Service robotcomprises a mechanical element 101A including one or more emitters 101Bthat emit automatically a localization signal to assist coupling of acarrier element 102C (of the transport robot 102) with mechanicalelement 101A (of service robot 101). The localization signal is receivedautomatically by one or more sensors 102A attached to transport robot102. Optionally, service robot 101 comprises one or more sensors 101Cand transport robot 102 comprises one or more emitters 102B to furtherassist the orientation and distance between mechanical element 101A andcarrier element 102C and thereby assist the automatic coupling. Onceautomatic coupling is complete, transport robot 102 moves automaticallyservice robot 101 to a new location using a motor 102D, such as one ormore rotary wing, one or more marine motors, one or more wheeled motors,and the like. The new location may be work surface needing servicing, amobile ground station for recharging/resupplying, a base station forrepairing, and the like. Optionally, transport robot 102 assistautomatically in the servicing, such as by applying fluids, removingdust, and the like.

Reference is now made to FIG. 2, which is a flowchart 200 of a methodfor coupling automatically between a transport robot and a servicerobot. A transport request is automatically received 201 by transportrobot 102, which automatically computes 202 a location of service robot101 to be moved. Transport robot 102 automatically moves 203 to thecomputed location, where a localization signal automatically sent 204 byservice robot 101 is automatically received by transport robot 102 toassist in coupling 205 between mechanical element 101A and carrierelement 102C. When coupled 205, service robot 101 is automaticallytransported 206 to a new location, such as a work surface, base station,mobile ground station, and the like, and automatically released 207.

Reference is now made to FIG. 3, which is a schematic illustration of acarrier element 300 for coupling between a transport robot and a servicerobot. Shown are a perspective view 300A, a side view 300B and a frontview 300C. Carrier element 300 comprises an opening 301 for aligning themechanical element and a female structural element 302 for coupling tothe mechanical element. Carrier element 300 may comprise guiding members304, optionally incorporating sensors, for guiding a male member of amechanical element into opening 301 and into female structural element302. Carrier element 300 may comprise sensors, electrical contacts,fluid conduits, and/or the like 303 for automatically positioning andaligning carrier element 300 with mechanical element. Carrier element300 comprises a base 305 for attaching carrier element 300 to transportrobot 102, such as on the underside of transport robot 102. The couplingmay allow power from transport robot 102 to automatically power thesensors on the coupling elements, and any actuators. It also allowsinformation from sensors to be transmitted to the UGV and UAV.

Reference is now made to FIG. 4, which is a schematic illustration of amechanical element 400 for coupling between a transport robot and aservice robot. Mechanical element 400 comprises a base 401 to attachmechanical element 400 to service robot 101. Mechanical element 400 maycomprise an optional articulating joint 402 which may allowautomatically orienting service robot 101 to a work surface 110orientation. For example, the articulating joint separates themechanical element into two parts, one proximal to service robot 101 andone distal to service robot 101.

Articulating joint 402 may be set at a fixed/passive orientation forwork surfaces 110 that may be at the same orientation, such as solarpanels, or dynamically/actively articulated for work surfaces 110 thatmay be at the different orientation, such as a vehicle/vessel hull. Forexample, an actively articulated joint may set a cleaning robot on eachmirrored surface of a concentrated solar power plant at an orientationdetermined in real-time as the angle of the mirrored surface. Forexample, a solar power plant has PV panel work surfaces at fixed anglesand the articulating joint is set a predetermined angle thatsubstantially matches the angle of the solar panels. For example, apassive orientation articulating joint has a fixed angle element and aflexible angle element that together allow for a range of angles of thearticulating joint, thereby allowing slight differences between theangle of the service robot and the angle of the surfaces. For example,the articulating joint has a combination of fixed, flexible and activelyarticulating elements.

For example, passive articulation allows some flexibility in thearticulation angles for safe orienting. For example, as the servicerobot is automatically lowered onto the work surface, one or more of itswheels or tracks or bottom may touch the work surface. An appropriatesensor on the bottom of the robot will indicate this. The transportrobot may then continue to lower the service robot to the work surface,until all the “bottom” sensors indicate that the service robot istouching the work surface. Then the transport robot will automaticallyreceive a signal from the service robot, through the coupling elements,that it is safe to release the service robot. For example, four distancesensors on the service robot, such as one in each corner and the like,indicate that the service robot is completely in contact with the worksurface. For example, when passive articulation is used the armmechanical element has electrical connections that transfer sensorsignals from the service robot's sensors to the transfer robot.

Mechanical element 400 may comprise magnetic field sensors and/oremitters 403, proximity sensors 404, and visible light signal emitters405 for assisting in automatically positioning and/or orientatingcarrier element 300 with mechanical element 400 such that larger malestructural element 406 may couple with a smaller female structuralelement 302, thereby allowing transport robot 102 to couple with servicerobot 101.

Optionally, carrier element 300 and/or mechanical element 400 may haveelongated portions, such as elongated bodies, to allow safe coupling.

Reference is now made to FIG. 5, which is a schematic illustration of acoupling 500 between a transport robot's carrier element and a servicerobot's mechanical element. Larger male structural element 406 is showncoupled 500 to a smaller female structural element 302, thereby allowingtransport robot 102 to couple with service robot 101. Guiding members304 form an opening to assist in automatically guiding the malestructural element 406 into the female structural element 302.

Reference is now made to FIG. 6, which is a schematic illustration of anisometric view of a service robot coupled to a transport robot's carrierelement 300 using the service robot's mechanical element 400. Mechanicalelement 400 is attached to service robot 101 and carrier element 300 isattached to transport robot 102 with mechanical attachment 601.

Reference is now made to FIG. 7, which is a schematic illustration of afront view of a service robot 101 coupled to a transport robot's carrierelement using the service robot's mechanical element. Service robot 101is coupled to mechanical element 400 using an orientation adjustmentelement 701. Mechanical element 400 is coupled to carrier element 300,such that energy and/or materials may be automatically transferredbetween the coupled robots. For example, a service robot transfersenergy to the transport robot through the coupling. For example, atransport robot may use sensors of a service robot during flight,pickup, drop-off, and the like using the coupling. For example,transport element may automatically adjust orientation, shape, center ofmass, size, moment of inertia, and the like using the coupling to maketransport of the robot pair easier.

Example applications of collections (“fields”) of physically-separatedsurfaces that may require frequent or regular inspection, diagnostics,maintenance, monitoring, or cleaning collectively referred to as“servicing”. Such examples include (but are not limited to):

-   -   Solar photovoltaic (PV) power plants, which may be composed of        two or more solar PV panel surfaces, forming rows or cells of        various sizes, physically separated from each other yet part of        the same power plant (such as a solar PV field. These PV panels        may be inspected and cleaned from dust on a regular basis for        the power plant to work at full capacity.    -   Windows on a single or multiple buildings form a collection of        physically-separated surfaces that may be cleaned frequently for        the buildings and windows to be visually attractive, and for the        windows to be functional (i.e., allow the designed amount of        transparency).    -   Concentrated solar power (CSP) plants may be composed of many        mirrors. Each flat or curved mirror in may be require regular        service to work at maximum power capacity.    -   Outer hulls of ships, airplanes, and vehicles to function        correctly (e.g., to reduce friction with air or water), these        hulls may be periodically cleaned, de-iced, treated with        appropriate materials, and the like. The work surfaces may also        be inspected for cracks, fissures, material faults, or other        damages that may interfere with their operation. The work        surfaces may be serviced to maintain their visual appeal (e.g.,        painted, cleaned).    -   The blades of air/wind turbine generating electricity constitute        surfaces that may be separated from each other. These require        regular servicing to maintain efficient power production and        prevent faults.

To clarify, while the examples herein refer to surfaces that vary onlyin shape, size and placement, the invention also covers fields whichcontain surfaces that vary in other characteristics.

There are many examples of surface collections, such as solar fields, ofphysically-separated work surfaces that may need servicing, such asfrequent or regular inspection, maintenance, monitoring, or cleaning,and the like. For example, solar photovoltaic (PV) power plants may becomposed of two or more solar PV panel surfaces, forming rows or cellsof various sizes, and physically separated from each other yet part ofthe same field. The solar panels may be automatically inspected andcleaned from dust on a regular basis, for the power plant to work atfull capacity.

Automatic machines, such as robots, may be built to service a singlesurface, but it may be difficult to build a robot that may effectivelyservice a single surface and move between surfaces within the field.Especially when the multiple surfaces are physically separated. Toovercome the difficulty due to the mechanical and energy requirements ofrobot movement between surfaces, a coupling assemble between transportand service robots may be used.

Method actions of aspects of the embodiments described herein may beperformed autonomously and/or automatically be preconfiguredinstructions to the robots. Optionally, the robots automatically storeprevious records of autonomous interactions to automatically applycorrective instructions to future interactions, such as changes tospeed, acceleration, distances, and the like. For example, when theautomatic release of a service robot by transport robot results in anaudible sound above a threshold level, the distance sensors used forautomatically performing the release may be calibrated for a smallerrelease distance value. Optionally, the robots are autonomous inperforming their tasks, such as by automatically performing series ofsteps according within each autonomous scenario. For example, the robotsare autonomous in their collective performance. Optionally, the robotsare partially autonomous in performing their tasks and partiallyoperated remotely by a human operator. Optionally, the human operatormay override the autonomous behavior of a swarm of robots, such as basedon heuristics, experience, standard operating procedures, safetyrequirements, and the like. Optionally, the automatic method actions maybe overridden by a human operator, and recorded by the robots to improveperformance and safety of the work surface, robots, tasks, and the like.

Following is a glossary of terms.

The term “robot” is used here to refer to an unmanned vehicular robotand/or system, which moves in/on air, water, ground, vacuum, such as inouter space, any combination of these, and/or the like on its own power.The system may be remotely guided by a human being, autonomous(automated), or semi-automated.

The term “service” is used to describe any operation taken by a robot tomaintain, diagnose, monitor, inspect, improve, repair, or otherwisebenefit a surface or another robot. For example: cleaning, applyingchemical treatment remotely or by physical contact, imaging using wavefrequencies in the visual or non-visual spectrum (e.g., includingthermal and other forms of infra-red imaging), taking measurements usingsensors or payloads (e.g., for inspection purposes), deploying payloadsthat relate to the recipient of the service (robot or surface or both),passing communications back and/or forth, transferring power (e.g., forelectric charging and recharging), fuel (and refueling), or materials toand/or from the recipient, establishing a physical connection to therobot or surface, fixing damages or faults, etc.

The general term “servicing” includes also “recharge”, used to describethe operation of docking a robot into a charging station, i.e., placingit sufficiently close to a specialized device where it may be refueled,electrically recharged, or otherwise have power transferred to it sothat it may engage in later operations; optionally, the device may alsoprovide storage, protection from the elements, etc. Recharging may alsotake place using a physical connection to a different robot, i.e., whenone robot recharges another. Regardless of where and how the rechargingoperation takes place, for the sake of brevity, the term “chargingrobot” may refer to both the robot or station transferring energy.

The general term “servicing” includes also specifically “monitoring” and“diagnostics”, is used to refer to identifying the status of a componentor sub-system or an entire system, along any scale, directly measurableby sensing devices, or indirectly measurable by processing the outputfrom sensing devices and/or combining such outputs. Optionally,monitoring includes also computation to differentiate nominal (correct,acceptable) states, from fault (abnormal, unacceptable) states.

A “service robot” may be adapted to apply a service as defined here intoa surface.

A “transport robot” may be adapted to transport a service robot. It mayalso transport other transport robots. Transport robots may also beservice robots, in the sense that they may also may be adapted to carryout services on surfaces or other robots.

The term “safe” is used to describe operations which do not harm ordamage robots and surfaces involved, nor other objects.

The term “pickup” is used to describe the safe and controlledestablishment of a physical connection between multiple robots, suchthat any optional subsequent movement of one or more of the robots (the“transport robots”, one of which at least is a transport robot), movesalso the other robots (the “service robots”). All robots together form a“pack”. To clarify, a pack is not formed by a transport robot and apassive object, such as a surface. It may only be formed between robots.

Inversely, the term “drop-off” is used to describe the safe andcontrolled disbanding of a pack, by safe and controlled disengagement ofthe physical connection between robots, such that all robots involved inthe pack may continue to move independently of each other.

The term “surface service task” is used to describe the operation of oneor more robots carrying out service of a given surface.

The term “transport task” or “transport” is used to describe a sequenceof safe and controlled pickup, followed by motion of the resulting pack,and drop-off. While the pack is in motion, the robots may also engage ina service operation in either one or both directions.

The term “pack service task” is used to describe a sequence of safe andcontrolled pickup, a service operation between robots in the pack (ineither one or both directions), and drop-off.

The term “transport robot task” is used to describe the safe andcontrolled motion of an individual transport robot, unconnected toothers (not part of a pack).

The term “coupling mechanical element” is used here to refer to theelements or mechanism facilitating physical connection between robots ina pack.

The term “arm” or “arm mechanical element” is used here to refer to thearm-like or mechanism facilitating physical connection between robots ina pack.

The term “carrier” or “carrier mechanical element” is used here to referto the receiving element of the arm or mechanism facilitating physicalconnection between robots in a pack.

Coupling

The challenges of picking up, dropping off, servicing, and moving arobot by another robot may not be the same as picking up, dropping off,servicing and moving with a passive cargo (e.g., a package):

For example, robots of different types may require special hardware andsoftware to coordinate their actions. For transport and pack servicetasks, all robots may be present at the same location, at the same time,and be at a state of operation that is adapted to the joint tasks,without harming the surfaces or stations. For example, for a robotpickup task, the transport robots may locate and identify the servicerobots, and position themselves such that the service robot may beconnected using the coupling elements, without applying damagingpressure on the surface. For example, transport robots may not add theirown weight to the weight of the service robots already on the surface.

For example, during a robot drop-off, the transport robots may locateand identify the target surface or station, and may position the servicerobot precisely on the surface, without applying damaging pressure onthe surface. In other words, transport robots may not add their ownweight to the weight of the service robots when reaching the surface,and may not allow gravity or natural forces to move the service robotsto the surface (e.g., service robots may not be dropped onto thesurface, even from a small distance).

For example, for the transport and pack service operations to takeplace, coupling elements may are used to actively form the physicalconnection, such that transport and service robot may move together, andto coordinate their pickup/drop-off/service such as through transfer ofdata values through the mechanical elements. For example, the mechanicalelements comprise digital and/or analog electrical connections fortransferring the sensor information, location information, and the like.

Following is a description of a robotic servicing system.

Optionally, a robotic servicing system is composed of multiple robots,one or more control stations, one or more charging stations (which mayalso be the control stations) and one or more base-stations for storingrobots (that may also serve as charging stations or control stations),that may be used for servicing surfaces. The system may be composed ofseveral components:

-   -   Transport robots.    -   Service robots.    -   Base stations where robots may be serviced, recharged        (refueled), and resupplied as necessary.    -   Control stations.    -   Repair stations.    -   Mobile ground stations.

The components may form packs of robots. The robots may carry outcomputation to process inputs, make decisions, communicate to each otherand/or to control stations and/or base stations.

Control/base/charging stations where manual or computational processesmay run to control and monitor the field, surfaces, other stations,and/or robots. The stations may also communicate with each other andwith robots, and may carry out computation on their behalf.

The system may employ input from the robots and additional sources ofinformation to identify the state of the surfaces (historically,currently, and projecting ahead). Likewise, the system may employ suchinput from the robots and other sources to identify the state of therobots, base stations, and control stations. The identified states maybe presented to a human operator in the control stations, or may be usedas input to automated processes.

Based on the information of (current, historical, and projected) states,the system may make recommendations as to which robot does what, andwhen, based on optimization criteria. The recommendation may be adaptedby the system autonomously, or it may be presented to a human operatorfor review, modification, execution and/or approval towards execution.This presentation may also be made at the control stations. The systemmay adjust (or allow a human operator to adjust, locally or remotely)its optimization criteria.

The control stations may allow a human operator to monitor theautonomous operation of the robots, to monitor the state of the robotsthemselves, to intervene in the robots' operation by directing,commanding, or tele-operating them, to monitor the surfaces beingserviced, and to direct the system to switch between modes of operation,as detailed below. The operator may revise, add, or remove optimizationcriteria from consideration.

Automatic machines (robots) may be built to service a single surface,e.g., in the examples herein. However, it may be challenging to build arobot that may effectively service a single surface, and move betweensurfaces within the field, so that it may service multiple surfaces thatare physically separated. This difficulty may be because of theenergetic requirements of movement between surfaces, which takes awayfrom the energy remaining for service, or it may be because motionbetween surfaces requires mechanical devices that may not, or areinefficient, or even hinder motion on the surfaces (e.g., moving inwater between hulls of ships).

Described here is an approach for two or more types of robots tointeract for servicing fields containing multiple separated surfaces. Aservice robot may service a single surface or multiple surfaces that maybe sufficiently close to each other so that it may reach them on itsown. A transport robot may be capable of transporting one or moreservice robots or other transport robots between surfaces which theservice robots may not reach on their own. The transport robot may alsotransport service robots and other transport robots from and to basestations in which the service robots and other transport robots may beserviced, refueled or recharged, maintained, and/or stored. In this way,the separation between surfaces may be overcome, without expensive andcomplex robots that may both service a surface and move themselves fromone surface to the next. That said, transport robots may assist theservice robots and/or service the surfaces themselves.

A robot system may contain several different types of robots, where afirst type of robot is intended to service the surfaces, and other typesof robots may transport or service the first type and/or each other.Types of robots are distinguished by one or more of their characteristicshape, mechanical features, computational capabilities, sensingcapabilities, payloads, actuation capabilities, mobility in, or on theboundaries of different mediums (such as air, water, ground, vacuum, andthe like), and their ability to service other robots or surfaces.Transport robots may be distinguished by their capability to transportone or more other type of robot.

Positioning includes at least three processes: pack formation (pickup),pack disbanding (drop-off), and pack motion (i.e., in a transportactivity). These processes may require transport robot and servicerobots to participate for the tasks to be carried out in a controlledand safe manner.

In some embodiments, the safety of the surfaces is considered. Forexample, when a pickup occurs, the transport robot may not applydamaging pressure to the surface, and under most applications, istherefore forbidden from touching it, as the mass of a transport robotis too heavy for the surface to bear. For example, when a drop-offoccurs, the service robot may be positioned exactly on the surface; itmay not drop onto it utilizing gravity, winds, or currents. Otherwise,the force resulting from the mass of the service robot hitting thesurface with the acceleration from natural forces may damage thesurface. Or, such forces may even move the service robot away from thesurface (for instance, in dropping off a hull service robot on a ship'shull, actual contact may be made between the service robot and the hull,or currents may carry it away; likewise, with window-cleaning robotsbeing positions on windows, gravity may pull them down unless theycontact the window). For example, when a transport occurs, the robotsmove together under the power of the transport robots. The mechanicalcoupling of service robots, such as mass, shape, and the like, to thetransport robots affects the ability of transport robots to controltheir movement, and thus may put the surfaces at risk of damage. Thisimportant constraint may be termed “surface safety constraint” or the“safety constraint”.

Some solutions may use proximity (or range) sensors on the transportrobot, measuring distance between a transport robot and the relevantsurface. However, these sensors may not accurately measure distance, andthus, the transport robot may not accurately identify its position withrespect to the target surface. This risks damaging the surface by eitherthe transport robots colliding with the surface, or causing a collisionbetween a service robot and the surface.

Following are a few examples of these difficulties below. For example, aUAV transport robot may attempt to use a downward-pointing laser rangefinder to measure distance to objects below it, such as surfaces,robots, protrusions, and the like. However, the laser range findersensor is susceptible to errors when used with glass surfaces, such astransparent PV surfaces. For example, laser sensors often see throughthem or when the work surfaces are translucent, the work surfaces mayreflect the laser back and give false readings.

For example, a transport robot may use ultrasonic sensors that measuredistance using sound/pressure waves. However, accuracy of sound wavesmay change with the angle of the surface to which distance is measuredand with respect to the angle of the sensor. For example, where thatangle is not 90 degrees, they measured distance readings may have verylarge errors, such as 10-20 of percent error.

For example, a transport robot may use infrared sensors to measuredistance using the intensity of reflected light in the infra-redfrequency range. However, the measurements of infra-red sensors changewith the temperature of the surface, and are also affected by theambient temperature. These factors may cause large errors in thedistance measured.

For example, a transport robot may use a touch-sensor connected to theend of a telescopic mechanical appendage or probe attached to thetransport robot, whose length may be changed dynamically. By extrudingthe prong towards the surface and measuring its length when the touchsensors indicates that the surface has been reached, the distance to thesurface may be measured. However, even here there are errors in theestimated length of the prong. In addition, there are significantlimitations on such prong, both in terms of limited range of operations,as well as when an external mechanical device affects the shape, mass,and maneuverability of the transport robot.

For example, a transport robot may use several cameras with overlappingview-fields to use the principles of stereoscopy to estimate thedistance of the surface from the transport robot. However, this dependson being able to identify common features in the images resulting fromthe cameras. A clean, all white surface, where no such features exist,an accurate measurement may be difficult. Additionally, a surface havingmany identical features may confuse the stereoscopic algorithms andresult in erroneous measurements. For example, when the corners andlines of photovoltaic panel surfaces look identical.

For example, a transport robot may use projected light patterns (such asin infra-red frequency ranges), and process images from a cameracapturing the projection to identify distortions, from which distancemay be measures. This method may be susceptible to external sources oflight, either artificial or natural, and may also be affected bytemperature of the surface and the environment.

In addition to the difficulties discussed herein, many methods mayprovide distance measurements at a single point. For example, toestimate the angle of a surface for a drop-off, when the service robotmay be aligned to the angle of the surface, multiple points may bemeasured simultaneously, and the service robots and/or transport robotsmay align themselves to the surface to avoid any impact. This may bedifficult or impossible (e.g., due to possible collision betweentransport robot and surface, or due to limited controllability by thetransport robot of the entire pack, e.g. under wind or currents).

Describe herein are procedures for a single transport robot (transportrobot) and a single service robot, but the procedure is easilyextensible to multiple transport robots and/or multiple service robots.

Following are detailed descriptions of major components of a roboticsystem.

Transport robots may be robots which carry other robots, using amechanical carrier element. Transport robots may include unmanned aerialvehicles (UAVs) which are capable of flight. Examples of such transportUAVs include but are not limited to fixed wing and rotating-wingaircraft, such as (but not limited to) airplanes, helicopters,quad-copters, hexa-copters, and octo-copters, and other vehicles capableof vertical take-off and/or vertical landing. Transport robots may alsoinclude unmanned marine vehicles (UMVs) which are capable of motion onwater surface, and/or underwater. Examples of such transport UMVsinclude but are not limited to unmanned boats, submarines, marinegliders, amphibian robots, and the like. Transport robots may alsoinclude unmanned ground vehicles (UGVs) which are capable of motion on ahard surface and/or underground. Examples of such transport UGVs includebut are not limited to legged robots, wheeled robots, tracked robots,combination platform transport ground robots (e.g. combinations ofwheels and tracks, etc.), and the like.

Service robots may be robots which service surfaces and/or other robots.Service robots may be coupled to transport robots using mechanicalcoupling elements.

Optionally, transport robots may assist in servicing the surfaces. Forexample, surface servicing is spraying of fluids on the surface. Anotherexample is on the use of rotating, moving, or fixed brushes to wipe thesurface. One example of a transport robot assisting the servicing of asurface is the use of water currents generated by a UMV to move dirtaway from the surface of a ship hull, or to loosen it. Similarly,another example is the use of the wind created by a UAV's down-draft toblow dust away from a panel, or using a UAV to spray cleaning materialson a surface. A different example is the use of a UAV to transfer powerto a UMV (this is an example of one transport robot servicing another).

Robots in general may execute computational processes by one or morehardware processor for their own control, and may also executecomputational processes on behalf of other robots, to generatecontrolling commands for other robots, and the like.

Optionally, robots may be constructed with their outer surfaces arecovered with materials that actively or passively reject dust, and/orallow recharging power sources from the light, heat, or chemicalinteractions with the environment in which they move.

All robots may be constructed such that they are protected from damagesfrom the environment.

To localize (position) themselves in relation to a target surface,robots may use standard sources such as satellite- or ground-based radiolocalization services (such as GPS, DGPS), or other sensory aids thatare specific to the target surface or the general area in which this andother surfaces are locations. For example, passive markers may respondto external power or radiation sources, such as color markers, lightreflecting markers, radio frequency identification (RFID) tags, and thelike. These passive markers may be placed in advance at variouslocations in the general area or on a work surface. For example, activemarkers may emit power or radiation, such as light sources (visible andinvisible), sound (human-audible or in-audible), electro-magnetic(radio) waves, and the like. These active markers may be placed inadvance on various locations in the general area or on a work surface.

By using active or passive sensing devices capable of sensing one ormore of these markers, a robot may localize itself within the generalarea and/or the target work surface. In addition, the robot may usesensing devices to track its position as it moves, by measuringdistances, acceleration, and the like. For example, reflection orelectro-magnetic field strength with respect to the surfaces and theground. For example, using the change in reflection from the surfaces tomonitor them, or localize on them. For example, by differentiating themarkers from the ground, and using images (in human-visible or invisiblefrequency spectrum) to identify specific locations. One example processused for such localization is called SLAM (Simultaneous Localization andMapping).

Robots may be protected from impact with the ground or surface, or fromsinking into the water in an uncontrolled manner, and from collisions,in a way that may not hinder their flight capabilities, and theircapacity for servicing and monitoring surfaces and other robots.Examples of such protections may include the use of parachutes, liquidbags, air bags, pressurized air or liquid, pumps, flotation devices,semi-rigid wire net, such as metal, plastic, or other material,surrounding the robots, and the like.

Robots may include communications equipment that allows them to becommanded, controlled, guided and/or tele-operated from the controlstations. Robots may also transmit communications on behalf of otherrobots, acting as routers or repeaters.

Robots may dock with stations, such as base stations, repair station,charging stations, and the like, that provide servicing, recharging,storage, etc. The stations themselves may have facilities for solar,wind, or current-based power generation, thus reducing or eliminatingtheir dependence on external power sources. The base stations are builtsuch that the transport robots may transfer service robots onto them(i.e., may position the service robots in docking), and/or the robots(of all kinds) may dock with them. When docked, robots may be servicedin multiple ways (for example, cleaned and recharged at the same time),or may simply be stored.

The docking mechanism may utilize gravity and other environmental forces(e.g., wind, water currents) to assist in securing the docking so thatrobots may dock while spending less of their own energy in docking. Forexample, a UGV robot that has a drained battery may dock with the basestation for servicing and recharging.

The docking mechanism may allow identification of the vehicle that iscurrently docked. The base stations may include one or more dockingpositions, so that more than one robot may be able to dock and/or storedat the same base station. The base station may close and seal itselfagainst dirt, wind, water, humidity, dust, sand or any contaminate, toprotect the docked vehicles are protected from it. The base station mayalso be manually closed and sealed.

The base stations may communicate to the control stations, and to eachother, and may report on their status. For example, what vehicles aredocked, what is the status of each docked vehicle, what time a dockingoccurred, predictions about what time servicing of each docked vehicleis likely to terminate, and the like. For example, how many open docksare currently available for docking with vehicles, how many docks arecurrently used, the types of robots that may be docked, what servicesmay be provided to each docked robot, what services may be provided inthe open docks, and the like.

Control stations serve to command, control, and coordinate all thedifferent systems discussed herein. The control stations may communicatewith the various robots, to command them, pass messages from any robot,base station, charging station, or control station to any other ofthese. A control station may also run computational processes on behalfof the robots, or run monitoring and mapping processes, analyzing theresults autonomously, and/or displaying them to operators.

The control station may receive data from the robots, surfaces, othersources of information, and various types of stations, to identify thestate of the surfaces, robots, and stations (historically, currently,and projecting ahead), either in real-time or not. The state of surfacesand robots and stations may include the identification of the locationand status of all robots of all types, relative to field of surfaces,service level, maintenance level, charge level, energy savings/outputefficiency, in real-time and non-real-time (e.g., statistics over windowof past), with identification of robot. The information may be used tocarry out diagnosis and failure detection processes, anomaly detectionprocesses, and other processing that pertains to the correct operationof the robots.

The identified robot states may be presented to human operators in thecontrol stations. One or more specialized displays and screens may beused to display information, state, and all process results (monitoring,mapping, diagnosis, fault-detection, servicing state, robot and stationstate, etc.) The displays may use any kind of visualization technique,such as displaying numbers and/or shapes and/or text and/or color tobetter convey the information to be presented to the human operators.

The control station software, the robots, and all stations may have anapplication programming interface (API) which allows addingcapabilities, transmitting new data from various sources, gettingavailable information to the control stations or any of the robots andother stations, and transmitting information.

The transmission of data from control stations to robots and vice-versamay take place either directly, from each robot to all others, and toall stations, and vice versa/The robots may serve as message routers, toimprove the communication capabilities and/or bandwidth requirements. Inthis way, a message from a robot or station A is transmitted to B notdirectly, but through one or more intermediate robots/stations, such asCl . . . Cn.

Overall, the objective of the system may be to efficiently servicesurfaces, by relying on transport robots to carry service robots to/andfrom stations and in-between surfaces. The transport robots may alsoservice each other, but this is a secondary function. The system employsalgorithms that manage all components of the system: the robots and thevarious stations. Optionally, the algorithms also manage the surfaces,relying on their identified state as aggregated in the control station.The algorithms may consider all or a subset of the processed andnon-processed information available to the robots and the variousstations, from any component of the system, external informationsources, and the surfaces. The algorithms may consider any subset ofpast information of the above, and including in addition previousrecommendations made, and the responses of the human operator and anysystem component (robots, stations, surfaces).

The algorithms' output may involve a recommendation as to the futuremotion trajectories of the robots, the future positions of robots, theassignments of tasks (service, monitoring, docking, etc.) to robots andstations, modes for the various systems (e.g., a reset or defaultoperation mode, a mapping mode, a service mode of any type, etc.) Therecommendations may also include assigning robots to pick up or serviceother robots. Thus, multiple robots may collect, deposit (e.g., into thebase stations), and service other robots.

These recommendations may span multiple overlapping time horizons, frommilliseconds to many days. The recommendations may be made available tothe stations and robots as needed, and be accessible (through a displayor other means of human-readable communications) to the human operators,such as for remote operations. The recommendations may also beautomatically or semi-automatically executed by the system. The humanoperator may intervene, modify, cancel, approve, revise therecommendations at any time.

In creating the recommendations, the algorithms may consider the serviceand monitoring needs of robots, surfaces, and stations. They may basethe recommendations on the needs for service and/or monitoring of any ofthe components, on the past or present commands of a human operator orhis/her wishes (as communicated to the system via some human-machineinterface), on possible detected faults or anomalies (and/or theirdiagnosis), weather information and other external information sources,etc.

Transport robots may service and pickup/put-down service robots. Themechanism that allows this, such as a mechanical element, may beattached to the transport robot and the service robot. The may alsoinclude reciprocal mating elements on the other robot, such as male andfemale structural elements. The coupling elements allow the robots toform a pair, which is coupled securely, such as sufficiently for motionbetween surfaces, or between a surface and a station. The mechanicalelement may not require active (powered) participation from the servicerobot so it may therefore work even when the service robot has no orlittle power left.

When a service robot is connected to a transport robot via couplingelements, it may be serviced by the transport robot, for example gettingrecharged or resupplied with servicing materials (e.g., cleaningmaterials). The inverse direction, service robot servicing the transportrobot, is likewise possible. For example, a service robot may transfermaterials or power to a transport robot.

The coupling elements are activated by the transport robot, or by theservice robot, or both. It may also be triggered (to lock or release) bythe docking stations, so that when a transport robot brings a servicerobot into a docking station or a base station, the locking mechanismmay be automatically released and/or an appropriate signal may be sentto the robots and docking (or base) station.

The system may utilize the robots to identify and locate target surfacesand general areas requiring servicing or more detailed monitoring. Suchidentification may also be made on a continual, persistent, or repeatingbasis to provide monitoring of the surfaces, identifying changes totheir state. To do this, the system, optionally through the controlstation(s), may:

-   -   Commands the transport robots to move at an altitude (for UAV)        or depth (for UMV), or distance that extends their range of        sensing to cover larger areas, as appropriate.    -   Command the robots to report sensor readings which may indicate        the need of surfaces for servicing. The sensor readings (e.g.,        images) may be transmitted or otherwise communicated, in        real-time or not, to the control station. There, automated means        or human input may be used to evaluate the need for service in        the sensed areas.

The procedure above may also be used to identify physical damage ortheft of surfaces, changes to their shape, orientation, position, etc.

To determine whether a target surface area requires servicing, anynumber of sensor modalities may be used, in different ways. For example,video footage and the use of image processing, visible orinvisible-light (IR, UV) cameras, lasers, sound responses (sonar), etc.Example methods that may be used to identify surfaces requiringservicing including sensing the electro-magnetic field, their light(visible and invisible) reflection, their heat reflection, soundresponse, etc.

There may be two issues in managing the fleet of heterogeneous robotsfor maximizing their ability to service surfaces:

-   -   Scheduling the pickup and drop-off activities: which transport        robots pickup which service robots, when and where (which        surface or station), and when and where (surface or station) do        they drop them off.    -   Positioning: The actual coordination of robots when forming a        pack (a pickup), disbanding a pack (a drop-off), and pack motion        (pack transport activity), such that they are safe (to the        surfaces involved and the robots).

Following is a description of the information maintained by the systemand provided to the algorithms used to plan, schedule, monitor, andcarry out the servicing of the surfaces in the field.

The field and surfaces contained within it may be representedgeometrically, logically, topologically, and the like. Geometricrepresentation of the field and surfaces means representing thetwo-dimensional (2D) or three-dimensional (3D) characteristics of eachsurface. For instance, a polyhedron (in 3D) or polygon (2D) is used torepresent the surface within a given limit of error. Another method maystore only the 2D or 3D coordinates of certain points on the surface, orof line segments that define the borders of the surface and theirlocation in space. The field may be represented by the scaled polygon(polyhedron) circumscribing all surface-representations, along with anyadditional associated structures (e.g., control stations, docking orbase stations, fuel and storage facilities, perimeter roads, maximumallowed altitude or maximum depth, etc.). Or it may be represented bythe collection of 2D/3D point coordinates that constitute the field, orthe parameters and/or coordinates of the line segments or geometricsurfaces that correspond to its internal structure. Each 2D or 3D pointin the field is uniquely assigned a point in the geometricrepresentation, and vice versa. A geometric representation is thus ascaled metric map (2D) or model (3D) of the field and surfaces.

A logical representation of the field and panels may involve assigning apotentially-unique identifier (string and/or number) to each panel andstation, to each objects and structure, such that given a point in thefield, or in its geometric representation, the identifier of the objectto which it belongs, if any, is known. Moreover, the identifier maydistinguish different objects: panels from roads from stations, etc.When the point is in free space (i.e., is not associated with anyobject), it may be assigned an identifier which distinguishes it asfree. Any point may be assigned an identifier which distinguishes it asbeing forbidden to move through, thus indicating that robots may notmove on a trajectory through this point. Points within objects may thusbe distinguished, but also free-space points might be assigned suchidentifiers (e.g., to represent zones where no robot may move, i.e.,“no-fly zones”). A logical representation of the field may be a labelingof every point in space, such that different objects and free space maybe differentiated and may be identified.

A topological representation of the field and surfaces involves creatinga mathematical graph, composed of vertices and edges, that representobjects on the field and their physical connections. Points in thefield, or in a geometric representation or a logical representation ofthe field and its constituent objects, which may be part of the sameobject, may be represented by a single vertex which is assigned anidentifier common to all of them. Such objects do not include freepoints. Thus, each vertex represents an object in the field. Additionalinformation associated with each vertex may include records of serviceor products of processing thereof, estimates of the duration of serviceunder various conditions, for each type of service, a projection of theexpected time until each type of service is repeated, a projection as tofailure time, etc. Vertices which represent intersections of motionroutes for transport tasks or pack service tasks may be a part of atopological representation of the field.

When a transportation route exists between two different objects in thefield (i.e., there is a sequence of adjacent free points that connectthe two objects, and that a pack may travel through), then an edge maybe added to the graph, connecting the two vertices representing the twoobjects. The metric length of the route may be noted on the edge, as mayother information pertinent to planning transport-packs and pack servicetasks that may make used of the route (e.g., width of road if on ground,curvature and absolute angle of route, allowed speeds of movement,accessibility to different types of robots, etc.). When motion isallowed on the route in only a single direction, the route is said to bedirectional, and the direction is associated with the edge. Otherwise,when motion is allowed in both directions between edges, then twodirectional edges are created, one in each direction. When multipledifferent routes exist between objects, they may be represented bydifferent edges between the corresponding vertices. The determination ofthe existence and characteristics of routes is described below. Edgesmay also represent physical separation between objects without thepossibility of robot motion between them, i.e., non-routes.

Determining which routes are feasible (exist) for pack transport tasksmay be done automatic, manual or a combination of these. The processgenerates feasible routes for all possible packs compositions, and fortransport robots. In general, several types of feasible routes may bedistinguished:

-   -   Routes for transport robots moving on their own from or to the        stations where they recharge, get serviced, or stored.    -   Routes for transport robots moving on their own from or to the        stations where service robots may be recharged, get serviced, or        stored.    -   Routes for packs (i.e., transport robots including one or more        transport robot, physically connected through coupling elements        to service robot) moving between surfaces, i.e., routes for        transport tasks where the starting position and goal position        are associated with surfaces.    -   Routes for packs (i.e., transport robots including one or more        transport robot, physically connected through coupling elements        to a service robot) moving between surfaces, and stations (e.g.,        for recharging) i.e., routes for transport tasks where the        starting position is associated with a surface, and the goal        position is associated with a station (or vice versa, i.e., a        route from a docking station for recharging to a surface, or        from a surface to a storage station).

Any feasible route thus distinguished may have additional informationassociated with it, also a product of the process generating the routes.Such information may include:

-   -   Safe speeds and/or altitudes and/or depths and/or direction for        the pack or transport robot moving it.    -   A mark that a route is not to be used for packs, only for        transport robots (or vice versa), or allowed for both.    -   A mark that a route may be stacked (i.e., transport robots        moving at different altitudes or depths may go in opposite        directions safely.    -   Safety margins in two or three dimensions, that denote the        spatial envelope within which motion is safe.    -   A unique identifier (string or number or combination) for the        route. This information may be associated with each route        separately for each type of robot, and for any composition of        packs.

For the system to carry out the scheduling of tasks and use of routes,information may be maintained about each robot. This information mayinclude:

-   -   The estimated 2D or 3D position of the robot at any given time.        The position may be represented in global coordinate system        (e.g., a GPS), in a coordinate system associated with the field        (i.e., where all positions are determined relative to an origin        point associated with the field), relative to a given surface        and an associated origin point.    -   The route identifier which the robot is currently traveling on        (if in a pack or a transport robot), or the object identifier        for the surface it is currently servicing (for a service robot),        or the station where it is stored, serviced, or recharged.    -   The type of robot, the services it provides, etc.    -   The current battery, fuel or power levels, including estimated        duration to depletion of such.    -   Fault information associated with the robot.    -   Last time in which it was known to communicate with the control        system, or any other robot.    -   A unique identifier.

Robots take on activities, such as tasks, services, and the like. Anassigned activity may be represented by a data structure containing thefollowing information:

-   -   The type of task involved (transport task, pack service task,        surface service task, recharge task, etc.)    -   For tasks involving service: The service(s) involved (e.g.,        application of a specific material, recharging, inspection,        diagnostics, monitoring, cleaning, etc.), and all details        associated with the specific service(s):        -   The expected duration of the service, the expected            termination time, the expected beginning time of the service            (if not already started) or the duration that the service is            already taking place.        -   The identifier of surface or robot being serviced, battery            being recharged, etc.        -   The identifier of the robots (or station) providing the            service, their power status (including current and predicted            power levels), etc.        -   The priority of the task on some absolute scale, or in            relation to other tasks pertinent to the same or other            surfaces.    -   For example, this represents the information pertinent to the        services associated with the task.    -   For tasks involving a pack the unique identifiers of the robots        in the pack, and each robot's role, the details of the physical        connection between pack members, the time in which the pack was        formed, and the time in which the pack is supposed to be        disbanded, the current and predict battery power levels and        duration until power reaches non-functional levels. In addition,        when the task is assigned a route, then all details of the route        (or a unique route identifier that be used to find the        information) may be stored with the activity as well. The        position of the pack (current estimated and predicted), priority        of transport (absolute or relative to other transports), etc.,        may also be stored.    -   For transport robot tasks: the unique identifier of the        transport robot, its current and predicted power levels,        beginning time of flight and remaining expected time of flight,        transport robot position (current estimated and predicted). In        addition, when the task is assigned a route, then details of the        route (or a unique route identifier that be used to find the        information) may be stored with the activity as well. Similarly,        when the task is assigned a priority, it may also be stored.

When an activity is not yet assigned to a robot, it may contain theinformation described above, except for the identities of the robotsinvolved (which are marked to denote that the robot(s) are not yet knownto the system). Likewise, when the route information is not yet known,then this information may be marked as unknown.

The objective of the system is to generate and assign activities torobots and routes such that when the activities are carried out close tothe beginning times stored in the activity data structure, theutilization of the robots and resources may be maximized, and the fieldmay be serviced as efficiently as possible, i.e., a maximal number ofsurface may be serviced by a minimal number of robots, and/or in a fixedtime duration.

Determine the full or partial order in which surfaces should be servicedmay be done by creating a schedule which yields effective service, withminimal idle on behalf of service robots waiting for their nextoperation to begin outside of their charging or docking stations. Such aschedule may allow flexibility, in that it may specify surfaces thathave equal priorities (in which case an arbitrary choice of ordering, ora choice based on other factors, such as remaining fuel or distance, maybe made). The scheduling process is triggered either automatically, orin response to manual command by a human operator, e.g., via the controlstation or remotely. Examples of automatic triggering of schedulesinclude (but are not limited to):

-   -   An automated process which considers meteorological data and        other data about surfaces to recommend services before or after        a storm.    -   A process considering the results of an automated or manual        inspection (a type of service) which identifies surfaces in need        of service    -   A process which determines the regularity (frequency) in which        surfaces are to be services.    -   The addition or removal of surfaces (e.g., a hull-cleaning        service might be needed when ships enter a port, or a        hull-inspection service may be needed when they leave).

In solar panel cleaning, the position and timing coordination may behandled opportunistically: The service robot requiring transportationmay signal a control system, and wait until the control system signals atransportation robot to pick it up. This may waste time while theservice robot waits, which could have been saved when the transportrobot would have already awaited it as it finished servicing the panels.Moreover, when two or more transport robots are used, then carefulscheduling of their motions along paths may be required, so they do notinterfere with each other, and to minimize transportation time.

In general, several algorithms may be applied to this scheduling task,generally falling under the umbrella term of ‘management of organizedbehavior’, such as those based on vehicle routing solvers, job-shopscheduling, the Hungarian algorithm and its extension for multiplerobots, and the like. For example, Beck et al describe an algorithm in“Vehicle routing and job shop scheduling: What's the difference”,Proceedings of the 13^(th) International Conference on ArtificialIntelligence Planning and Scheduling, 10-14 Jun. 2013. For example,Dantzig et al. describe an algorithm in “The Truck Dispatching Problem”Management Science, 6 (1), pp: 80-91, October 1959,doi:10.1287/mnsc.6.1.80. For example, Christofides et al. describe analgorithm in “The Vehicle Routing Problem”, 1979, Chichester, UK, Wiley.pp. 315-338. For example, Frazzoli et al. describe an algorithm in“Decentralized algorithms for vehicle routing in a stochastictime-varying environment”, Proceedings of the 43rd IEEE Conference onDecision and Control (CDC), 14-17 Dec. 2004, pp: 3357-3363 Vol. 4, DOI:10.1109/CDC.2004.1429220. For example, Psaraftis describes an algorithmin. “Dynamic vehicle routing problems”, Vehicle Routing: Methods andStudies, 1988, 16, pp: 223-248. For example, Bertsimas et al. describean algorithm in “A Stochastic and Dynamic Vehicle Routing Problem in theEuclidean Plane”, Operations Research, 1991, 39 (4), pp: 601-615,doi:10.1287/opre.39.4.601. JSTOR 171167. For example, Toth et al.describe an algorithm in “The Vehicle Routing Problem”, 2001,Philadelphia: Siam, ISBN 0-89871-578-2. For example, Wurman et al.describe an algorithm in “Coordinating Hundreds of CooperativeAutonomous Vehicles in Warehouses”, Proceedings of the NationalConference on Innovative Applications of Artificial Intelligence (IAAI),Jul. 22-26, 2007, pp: 1752-1760. For example, Chan et al. describe analgorithm in “The multiple depot, multiple traveling salesmanfacility-location problem: vehicle range, service frequency, andheuristic implementations”, 2005, Mathematical and Computer Modeling 41,pp: 1035-1053. For example, MacAlpine et al. describe an algorithm in“SCRAM: Scalable Collision-avoiding Role Assignment withMinimal-makespan for Formational Positioning”, Proceedings of theTwenty-Ninth Conference on Artificial Intelligence (AAAI), Jan. 25-30,2015, pp: 2096-2102.

For a pick-up to occur, the service robot and transport robots may bothbe present at the same time in sufficient proximity between them so thatcoupling elements may be used to physically connect them. In addition,because of the safety constraint, the actual physical connection may beestablished without harm to the surface.

Following are example stages of a pickup procedure embodiment.

A pickup procedure is initiated by a “pickup request” signal which issent from a control station, from a robot, and the like.

-   -   The signal may contain information about the location of the        robot to be picked up. This information may be in the form of        the geometric coordinates in the coordinate system used by the        field; or it may be in the form of a unique identifier of the        object (surface or station) where the robot to be picked up is        present.    -   The signal may additionally contain the unique identifier of the        transport robots which is to respond to the pickup request. This        is particularly important when the signal is transmitted to        multiple transport robots of which only one is to respond;        otherwise, multiple transport robots may utilize one of the many        readily available task-assignment or voting protocols to decide        on the transport robot which may respond.    -   The signal may additionally contain the unique identifier of the        robot which is to be picked up.    -   The signal may additionally contain information pertinent to the        motion of the transport robot towards the robot to be picked up,        such as route information, power status and time remaining,        requested velocity, obstacles expected, etc.    -   The signal may additionally contain information on any        subsequent tasks for the transport robot, such as on the        transport activity that may commence once the pickup procedure        completes, the drop-off task, etc.

The transport robot may initiate movement from its current position tothe position contained in the pickup request signal. The transport robotmay carry out this movement in a safe and controlled manner, usingreadily available methods to

-   -   plan the transport robot's route and trajectory avoiding        obstacles, surfaces, stations, and areas forbidden for its        movement. Such planning may be carried out by the transport        robot or any external computational device, e.g., in the        stations    -   carry out the motion along the planned route, maintaining any        safety or legal requirements for the motion (e.g., minimal        altitude, no-fly zones, maximum depth, etc.)    -   dynamically respond to contingencies along the route, such as        unexpected obstacles, dynamic motion of other robots, wind and        currents, etc. Responses may include re-planning the motion        trajectory and route, canceling when necessary, etc.

The specification of a goal location supplied by the pickup requestsignal may be sufficient for the planning and execution of a motiontrajectory for the transport robot. For example, a representation of thelocation within the field that matches the type of specification used(e.g., GPS coordinates and a geometric representation which uses GPS asthe coordinate system). The target location information may beapproximate, such as when the specification is necessarily inaccurate,as it is subject to errors in measurement and estimate, or onlyspecifies an area in which the picked-up robot is to be found (e.g.,when specified by an identifier of an object such as a surface). Thisstage may end when the transport robot arrives at the approximateposition as provided in the pickup request signal, or alternatively,when the pickup procedure is aborted.

When the transport robot arrives at the location specified in the pickuprequest signal, the transport robot and the robot to be picked up(henceforth in the description of this procedure, “service robot”) maynow locate one the other, or mutually, so that coupling elements may beused to form the actual physical connection between transport robot andservice robot. For example, one or both robots may move such that theirrespective position is within the definition of “catching zone” definedby the carrier and arm mechanical element utilized, and the guidingframe elements incorporated.

Two cases may be distinguished. In the first case, a uniqueidentification of the service robot is possible, distinguishing it fromother robots and from the surface on or station in which it is located.In the second, the service robot may not be distinguished from otherrobots. In the first case, when the transport robot successfully is inthe approximate location of the service robot, the transport robot mayidentify its relative location with respect to the service robot usingone or more sensors. For example, the relative location may be specifiedor translated into polar coordinates where the transport robot is at theorigin point, and a bearing and distance to the service robot areidentified and/or computed.

Examples of such identification procedures may include:

-   -   Use of cameras, whereby the transport robot visually identifies        the service robot on the surface or in the station, in an image        taken of the surface or station. The translation from image        coordinates (the pixels in which the service robot is in the        image) to relative location may be straightforward, which may        themselves require information which may be readily available        (such as the altitude of a UAV transport robot, its approximate        distance to the surface, etc.). There are many different known        ways of using cameras for this purpose. A few examples include:        -   Computer vision recognition algorithms to identify the            service robot, its arm mechanical element, or any visual            markers on the robot or arm.        -   Visually identify light projections unique to the robot,            where such light projects are activated by the service robot            or by its arm mechanical element (see Section 3). For            instance, the service robot may shine a light in an agreed            upon frequency, or may turn it on and off in some            predetermined regular pattern, much like lighthouses use            unique light signatures for identification. To identify such            patterns, multiple images are taken by the transport robot            of the general area in which the service robot is            hypothesized to be. By using simple background differencing            or more advanced techniques, the pixels changes from one            image to another pinpoint the image coordinates of the            service robot.        -   The use of multiple methods described above (e.g., four            different lights) may not only give the position of the            service robot with respect to the transport robot, but also            its pose (e.g., angle on the surface or in the station).    -   Use of one or more microphones on the transport robot to        identify predetermined auditory signals (predetermined to not be        confused with expected ambient sounds) emitted by the service        robot or its arm mechanical element. Such signals may be used        predetermined frequencies or auditory patterns, or combinations        thereof; they are not necessarily in frequency ranges that        humans may hear. Sound localization devices and procedures may        be used for this purpose. Here, the microphones may provide at        least the bearing of the relative location.    -   Use of RFID localization methods to identify a service robot or        its arm mechanical element which contain RFID markers.

Identification procedures may be used by the service robot to identifythe transport robot (directly, or through its arm mechanical element),and to position itself accordingly (if possible), and/or to report therelative location of the service robot with respect to the transportrobot, so that the transport robot may move to compensate for anyremaining gap in positioning.

The transport robot and/or arm mechanical element(s) and/or servicerobot may adjust their position to reduce the difference in theirrelative locations, using motion control techniques for robots. Theprocess of identification of the relative location of the service robotwith respect to the transport robot continues iteratively until thetransport robot and service robot are position in the catching zone ofthe carrier mechanical element.

The second case is when the transport robot arrives at the approximatelocation of the service robot, but may not identify it uniquely, e.g.,through any of the means described above. This may occur because morethan one service robot is identified (for instance, two service robotsare visually identified, and may not be distinguished from each other inthe image), or because there no service robot at all may be identified(e.g., because of environmental conditions that interfere with theidentification process). In the latter case, where no service robot maybe identified readily, the transport robot may communicate its failureto the control station, and/or the service robot. One option in thiscase is to the abort the procedure—such a decision being taken by aprocess in the control station or a human operator, or by the transportrobot or service robot. Alternatively, a command may be sent from thecontrol station or from the service robot to retry the process,optionally with a different set of coordinates, or with a differentmethod of identification.

For example, the service robot may be commanded to flash lights on itsarm mechanical element in greater intensity or in different frequency,while a command to the transport robot is sent to instruct it tore-attempt identification using a matching method. Or in anotherexample, the transport robot may move to increase its distance from thetarget coordinates, such that images captured by its cameras capture alarger area in which the service robot could potentially be identified.

In the former case, where more than a single robot may be identifiedreadily, the transport robot may act as described in the handling thecase of no identified service robot. When the service robot changes itsidentification method, it may be distinguished from the other robots.Alternatively, it may also attempt to move towards one of the identifiedservice robots, e.g., the one closest to it, or arbitrarily chosen.

When not aborted, the transport robot and service robot may bepositioned within the capture zone defined for the carrier mechanicalelement in use. The transport robot and service robot may now follow thecapturing procedure appropriate for the arm mechanical element in use byaligning male and female elements on each respective robot.

When robot drop-off takes place, the robots forming the pack may bepositioned such that when they execute the release procedure for the armmechanical element used (i.e., the pack is disbanded), they may continueoperations in a safe and controlled manner, thereby avoiding damage toeither robots or surfaces. For instance, drop-off where the transportrobot is a flying vehicle (“a transport robot”), and the service robotis a ground vehicle, may be done such that the ground vehicle has landedsafely and without damaging itself, the transport robot, or the surfaceor object it has landed on.

The drop-off procedure may be initiated by a “drop-off request” signalwhich is sent from a control station, or from a robot, e.g., in responseto the termination of a transport activity in which the pack is engaged.

-   -   The signal may contain information about the location of the        surface or station for the drop-off. This information may be in        the form of the geometric coordinates in the coordinate system        used by the field; or it may be in the form of a unique        identifier of the object (surface or station) where the service        robot is to be dropped off.    -   The signal may additionally contain the unique identifier of the        service robot which is to be dropped. This may be important when        the pack is composed of multiple service robots.

The signal may additionally contain information on any subsequent tasksfor the transport robot or service robot, such as on the transport robottask that may commence once the drop-off procedure completes, etc.

Optionally, the drop-off location is predetermined before the pickup.

The pack may already be maximally near the target surface or station,without risking damage to the robots or surface. This may happen, forinstance, when the end of a transport activity left the pack in thisposition. The pack may initiate its movement from its current positionto the position contained in the drop-off request signal. The transportrobot (acting to move the pack) may carry out this movement in a safeand controlled manner, by:

-   -   planning the pack's route and trajectory avoiding obstacles,        surfaces, stations, and areas forbidden for its movement. Such        planning may be carried out by robots or any external        computational device, e.g., in the stations    -   moving along the planned route, maintaining any safety or legal        requirements for the motion (e.g., minimal altitude, no-fly        zones, maximum depth, etc.)    -   dynamically responding to contingencies along the route, such as        unexpected obstacles, dynamic motion of other robots, wind and        currents, etc. Responses may include re-planning the motion        trajectory and route, canceling when necessary, etc.

The new computed location supplied by the drop-off request signal issufficient for the planning and execution of a motion trajectory for thepack. For example, by a representation of the field that matches thetype of specification used (e.g., GPS coordinates and a geometricrepresentation which uses GPS as the coordinate system).

That said, it is understood that the target location information may beapproximate; the specification may be inaccurate, as it is subject toerrors in measurement and estimation, or only specifies an area in whichthe target surface or station is to be found (e.g., when specified by anidentifier of an object such as a surface). In this case, the pack maymaintain movement towards the target surface or station until it may nolonger do so, such as when an automatic process decides that (1) it maynot move further without risking damage to the surface or robots, or (2)it is within a given distance of the surface or station, where suchdistance is specified externally to the procedure.

The pack arrives at the approximate new location, and approaches thesurface or station as described, or alternatively, the pickup procedureis aborted.

The transport robot moves the pack to align the angle of the servicerobot with the surface, as closely as possible. This may be done, e.g.,by actively changing the relative angles between robots using the armmechanical element, or by changing the heading and angles of thetransport robot (and thus of the pack) such that the service robot isbetter aligned with the surface. The transport robot moves the packtowards the center of the target surface or station, while stillmaintaining the minimal safety distance to it. This may increase thetolerance of the positioning procedure to errors in localization of thepack with respect to the borders of the surface or station. It may notbe necessary to position the pack at the shortest possible safe distancefrom the center. Rather, it may be sufficient to position the pack suchthat the distance to the borders is within tolerance considered safe.

The transport robot moves the service robot towards the object (surfaceor station) in a velocity deemed safe for the robots and the object,i.e., such that impact caused by the service robot as it touches theobject may not cause damage. The transport robot may stop as soon as theservice robot is safely positioned on the surface; this may be detectedin one or more ways. For example:

-   -   The transport robot may feel a change in the acceleration or        velocity of the movement, caused by the friction or resistance        of the object to additional motion.    -   The service robot may use its sensors and/or sensors on the arm        mechanical element to determine when the service robot is safely        positioned, and inform the transport robot. For instance, the        service robot may use pressure sensors or short-range distance        sensors in the area which is in touch (or just above) the        object, and report their values to the transport robot, e.g.,        through the arm mechanical element, or independently. Familiar        sensor fusion techniques may be used to determine when the        reported values indicate safe positioning on the surface.

For example, a UAV transport robot positioning a surface cleaning robotmay use several pressure sensors within the wheels or tracks of theservice robot to identify when it is touching the surface. For example,multiple sensors in multiple locations may indicate all supporting partsof the robot are safely lying on the surface. The UAV may begin movingslowly towards the surface. When one or more wheels touch the surface,the appropriate pressure sensor readings indicate so. The readings arecommunicated through the coupling elements to the UAV. When all wheelsare on the surface, all readings may indicate safe support on thesurface, and the UAV may stop moving.

When no safe positioning is achieved within a given time limit, or whenunsafe positioning is reported by sensors, the procedure may be aborted.Alternatively, the transport robot may move the pack back in theopposite direction, away from the object, so that the procedure may beretried (the entire stage). Once the service robot is in the correctposition, the pack initiates the release procedure for the carriermechanical element to decouple from the service robot.

When traveling as a pack, the service robot may make motion of thetransport robot (and thus of the pack) more difficult. For example,sensors of the transport robots may be blocked from view by the servicerobots, or because the shape of service robots may interact with thesurrounding fluid motion, such winds and currents, and effect themotion. For example, the mass of the service robot effects the center ofmass of the pack, which is different than the center of mass of thetransport robot.

Several procedures may be used during pack motion to alleviate thesedifficulties:

-   -   First, the coupling elements may be used to transmit sensor        information from the service robots to the transport robots. For        example, a surface cleaning robot with a forward-facing camera        may transmit imagery through the coupling elements or through        other means to the transport robot, thus allowing the transport        robot a different angle of view to identify obstacles. Or a        downward facing camera in the cleaning robot may transmit        imagery which may be used to estimate the horizontal motion of        the pack, e.g., via familiar visual odometer procedures.    -   Second, the arm mechanical element may change the relative        position and angles of the transport and service robot, to        adjust the pack overall shape, its center of mass, to make        motion easier and less affected by environmental conditions        (such as winds and currents).

A mechanical coupling element is the term to refer to mechanicalelements of any coupling mechanism between the two robots, such as usedto form a pack, by physically, electronically, and logically connectingrobots. For example, this is different from a physical connectionbetween a robot (or vehicle) and payload. The mechanical element itselfmay be made from one or more elements, attached to one or more robots.

A coupling element has several important characteristics:

-   -   It may form a safe and reliable physical, mechanical connection,        whereby one or more robot connects to one or more other robot,        such that motion by the transport robots moves the connected        robots.    -   This, for instance, distinguishes coupling elements from other        mechanical element s used for vehicle-to-vehicle refueling,        e.g., in the air or sea (where both vehicles may coordinate        their movement for the refueling to take place). The physical        connection of a fuel line may not translate motions of one of        the vehicles into motions of the other vehicles.        -   The formation of the safe and reliable physical connection            may be called capturing. For each embodiment of a carrier            mechanical element, there exists one or more specific 2D or            3D area called a capture zone, which lies near the arm            mechanical element. When two robots are positioned within            their respective mechanical element in the capture zone,            they may engage in a capture procedure. Once the procedure            is run successfully, motion of the transport robot causes            motion of the service robot.        -   The disbanding of the physical connection is called release.            For each embodiment of carrier mechanical element, there            exist one or more specific 2D or 3D area called release            zone, which lies near the arm mechanical element (often,            identical to the capture zone. When transport and service            robots are positioned within the respective mechanical            element release zones, they may start a release procedure.            Once the procedure is run successfully, motion of the            transport robot is independent from the service robot.    -   The mechanical element may be used by two or more robots,        whereby one or more of the robots is autonomous, i.e., not        controlled by a human, locally or remotely. The deployment and        operation of the mechanical element is controlled by the        connecting robots, and they may autonomously adjust their        position or pose, or deploy additional processes and devices, to        aid in the forming of the pack via the coupling elements. After        the formation of the pack, the robots may further adjust their        pose or deploy additional processes and devices to assist in the        motion of the pack, e.g., to make it more efficient. For        example, distinguishes the coupling elements from an        inter-vehicle locking mechanisms, which depend on human        supervision and/or control to be deployed, e.g., when one        vehicle is towing another.    -   The mechanical element allows flexibility in the relative poses        of the robots in a pack, with respect to each other, so as        facilitate motion of the pack (e.g., to improve maintenance of        center of gravity, control or efficiency of flight, etc.). For        example, the arm mechanical element may be physically shaped to        maintain robots in a certain angle with respect to the direction        of movement of the pack, and/or the surface. In another example,        an arm mechanical element used to connect a UAV transport robot        and a UGV service robot and the arm mechanical element allows        passive (e.g., due to wind) or active (by the robots themselves)        change to the angle of the UGV with respect to the UAV, to        improve balancing and facilitate easier control of the flight.        In another example, an arm mechanical element used to connect an        unmanned boat (here, the transport robot) which is carrying a        hull-cleaning robot by flexibly dragging it in the water. The        water currents may cause changes in the position and angle of        the robot with respect to the boat, allowing for motion with        less resistance from the water.    -   The coupling elements may form at least two of three levels of        connection between the robots in the pack:        -   A mechanical connection, which may be used safely and            reliably for the joint motions of the robots.        -   A sensing and actuation electrical connection, allowing a            transport robot to utilize the sensors and actuators of a            service robot, or vice versa. For instance, a transport            robot transport robot may utilize the sensors on the            underside of a service robot to measure distance to surface            or floor; or it may use video imagery from cameras on the            service robot to stay clear of obstacles, estimate the pack            movement relative to the field, etc.        -   A communications connection, by radio-frequency, light, or            electrical means, which may allow the transmission and            receiving of information pertaining to commands, control,            monitoring of the pack, or to the motion of the pack, or to            the activities of the pack and the robots that constitute            the pack. Such information includes also the results of any            computation carried out by one robot for the benefit of            another, e.g., the video imagery in the example above may be            processed wholly or partially on the service robot before            being transmitted to the transport robot.    -   The coupling elements may allow multi-directional transmission        of information, commands, and control. Transport robots in the        pack are responsible for the physical motion of the entire pack,        but the control of the motion is not necessarily carried out in        any one robot, nor necessarily in the transport robot. It is        possible for a service robot to be in command and control of the        motion of the entire pack, as an example.

Aspects of embodiments that are suitable are a pair of matchingmechanical connectors, such that one of the pair has a bulbousprotrusion that is of a first size/shape, such as a male mechanicalelement, and such that the second of the pair has a large receivingregion, such as significantly larger than the first size/shape, such asa female element, to guide the protrusion into a carrier element thathas a second size/shape smaller than the first size/shape. Thus, theprotrusion can temporarily “lock” into the carrier for the transportrobot to carry the service robot. For example, the protrusion mechanicalelement is part of an arm mechanical element attached to service robot,and the carrier element is attached to the transport robot.Alternatively, the arm is attached to the transport robot and thecarrier is attached to the service robot. Alternatively, to a male andfemale mechanical carrier elements there can be two similarlysized/shaped mechanical elements, such as two hooks, each attached to arobot, such that one hook latches with the other hook. Similarly, otheroptions for latching mechanical elements of substantially the same sizeand shape can be designed.

Optionally, the protrusion can have an isotropic symmetrical shape, suchas a ball, so that the service robot may rotate freely within thecarrier mechanical element. For example, the protrusion is a ball withannular electrical connectors for transferring sensor signals,transmission signals, power, and the like. Optionally, the protrusionhas an anisotropic symmetrical shape, such as a brick with a curvedbottom, so that the service robot can rotate freely only on one axis,within the carrier mechanical element and the electrical connectorsaligned parallel to a curved edge. For example, a brick with a curvedbottom shaped protrusion has a single axis of rotation to allow theservice robot to align with an angled surface. For example, thehemispherical protrusion has annular electrical connectors and areceiving conduit on the center of the flat surface of the hemispherefor transfer of service materials, such as fluids.

Optionally, optical signals are used to transmit information, such asproximity sensor values, robot status, and the like, between the servicerobot and the transport robot. For example, a light emitter sends lightin pulses which are captured by optical sensors on the matching elementsand translated into digital value signals. Optionally, the protrusionhas multiple radial connectors, and the two sides of a funnel of thecarrier mechanical element are used to transmit signals, electric power,and the like. For example, two contacts on the protrusion are selecteddynamically, each one in electrical contact with one half of the carrierelement, thereby establishing a two-wire connection which can be used totransmit signals (e.g., by serial communications), electricity, and thelike.

In addition, the coupling elements may have additional characteristics:

-   -   The coupling elements may form a connection through which fuel        or electricity or power may be transmitted, for the purposes of        recharging one or more of the robots in the pack. Recharging may        take place in one direction (from one robot to others), or in        multiple directions (from one robot to others, from another        robot to the first robot and others, etc.).    -   The coupling elements may form a connection by which materials        used to service the surfaces or other robots may be transferred        (e.g., cleaning and maintenance materials in liquid or gas or        solid form).    -   The coupling elements may form an electrical connection such        that the coupling elements enhance radio frequency transmissions        and receiving for one or more robots in the pack.    -   The coupling elements may be powered autonomously, or may be        powered when connected to one or more robot, or may require no        power. It may have sensors and actuators or active markers on        board (such as lights), which are powered by the coupling        elements power source, whatever it is on the coupling elements        or from the connected robots. It may have one or more hardware        processor or controller on board to execute processor        instructions. For example, to carry out computational processes        for controlling the arm, reporting on its status, establishing        connections to the robots, identifying the robots, etc.    -   The coupling elements may be marked by non-powered means        (magnets, passive markers, including visual markings and        reflective surfaces, RFID) to assist in identifying the robots,        establishing connections to the robots, etc.    -   The coupling elements may be built to mechanically improve and        ease the forming of a connection between robots. As an example,        (here, of coupling elements in two parts) the physical shape of        the coupling elements (in the example, in two parts), is such        that the arm mechanical element part on one robot is        mechanically guided into the right position in the carrier        mechanical element part on the other robot. Alternatively, in an        arm mechanical element of a single part, the shape of the arm        mechanical element is such that a robot itself is mechanically        guided into the right position as above. This reduces both        mechanical shocks and control the positioning of the robots in        six degrees of freedom of movement (three dimensions for        position, three angles).    -   The coupling elements may be built to passively or actively        reduce shocks when robots position themselves to form a        connection.    -   The physical shape of the coupling elements may be fixed or may        change before and during the formation of a pack, and after a        pack is disbanded, to facilitate improvement in the formation        process itself, or in the disbanding process. Such changes may        affect the position and angle between robots, or between robots        and surfaces. The change may take place passively (e.g.,        mechanical responses to changes in pressure), or may be actively        controlled by the arm or robots.

Provided below is an example embodiment of coupling elements, toillustrate key characteristics of the arm as described above. This is adesign for coupling elements in two parts. One part may be attached to aUAV transport robot, capable of lifting and flying with a surfacecleaning robot (UGV). The other part may be attached to a surfacecleaning robot, such that when the two parts are connected, thetransport robot is a transport robot, the UGV is a service robot, andthe two form a pack which may take-off, fly, and land, together.

The capture procedure for this example, where the UGV is on a surface,works as follows. The UGV positions itself at the top of the surface,angled such that the ball is at the zenith (90 degrees straight up). Theangle of the arm is set to compensate for the angle of the surface tothe horizon, a benefit of the arm mechanical element as described above.The UGV transmits power to the arm mechanical element attached to it, sothat the light source shines light at a spectrum that may be detected bycameras on the UAV. The UAV uses this light to identify the approximateposition of the UGV, and hovers above it and slightly behind it, suchthat the capture zone, here marked opening is above the UGV, but forwardmotion by the UAV may be used to bring the opening to be above the ball.The UAV may use one or more downward facing camera to identify theposition of the ball with respect to the opening, whose position isknown to the UAV (by presetting). This identification is done bylocating the light source in the images originating in thedownward-facing camera.

The UAV then lowers altitude until one or more of the followingconditions occur: (1) the proximity sensors behind the opening report tothe UAV that the distance to the arm holding the ball is such that theball may be within the polygon defining the opening area; (2) acomputational process may identify that the ball, visible in an imagefrom a camera of the UAV, is within the polygon defining the capturezone (opening area); (3) the proximity sensors on the arm may report tothe UGV that an object is in proximity to the ball; (4) one or moreimages coming from a camera on the UGV and facing up towards the UAVdetect the UAV's presence and estimate its position such that the ballis within the polygon defining the opening area; (5) magnetic sensorsembedded in either arm mechanical element, carrier mechanical element,or both report to the attached robots that a magnet (embedded, e.g., inthe ball, or in the opening) is nearby. The communication of informationfrom sensors in the robots and in coupling elements is an illustrationof the benefit of the coupling elements.

Once the appropriate set of conditions hold, the UAV stops loweringaltitude, and moves forward a small distance, enough to trap the ballwithin vertical space above the ball catcher. The shape of the openingmechanically guides the arm holding the ball to the correct position.The proximity sensors may be used to verify that the ball is in place,and then the UAV goes up, possibly with some forward motion as well,trapping the ball and securing it in in the ball catcher. This forms aphysical connection which has some flexibility during flight (e.g., toallow some movement due to forces of wind acting on the UGV), but stillmay not allow the ball to leave the ball catcher, and thus disconnect.

An example robotic system using coupling assemblies may be composed ofmultiple unmanned vehicles (hereafter: robots), one or more controlstations, and one or more base-stations, that are used together for agoal, such as maintaining, servicing and monitoring work surfaces, suchas solar panel surfaces and the like. The system may be composed of oneor more aerial (flying or hovering) robots (hereafter: UAVs), capable ofvertical liftoff and landing, that may service, maintain and/or monitorthe work surfaces. Optionally, the system comprises one or more marinerobots (hereafter: UMVs), capable of moving on water and/or underwater.The UMVs may also be capable of flight. UMVs may be able to maintain,service and/or monitor work surfaces they are in contact with in amarine environment

The system may also comprise one or more other robots (hereafter: UGVs),incapable of flight, which may maintain, service and/or monitor worksurfaces they are in contact with. Base stations may be included in thesystem where the UAVs, UMVs, and UGVs may be serviced, recharged(refueled), and resupplied as necessary. The UAVs and UMVs may have thecapacity to transfer the UGVs and/or UMVs between different worksurfaces and/or between a given work surface to a base station. The UAVsand UMVs may also service (and/or recharge, resupply, and the like) theUGVs and other UMVs. When so, UAVs, UMVs and UGVs may optionally havespecialized mechanisms for this purpose.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device havinginstructions recorded thereon, and any suitable combination of theforegoing. A computer readable storage medium, as used herein, is not tobe construed as being transitory signals per se, such as radio waves orother freely propagating electromagnetic waves, electromagnetic wavespropagating through a waveguide or other transmission media (e.g., lightpulses passing through a fiber-optic cable), or electrical signalstransmitted through a wire. Rather, the computer readable storage mediumis a non-transient (i.e., not-volatile) medium.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Java, Smalltalk, C++ or the like,and conventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,to perform aspects of the present invention.

Aspects of the present invention are described herein regardingflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products per embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general-purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

In the description and claims of the application, each of the words“comprise” “include” and “have”, and forms thereof, are not necessarilylimited to members in a list with which the words may be associated. Inaddition, where there are inconsistencies between this application andany document incorporated by reference, it is hereby intended that thepresent application controls.

1. A method for autonomous interactions between robots, comprising:receiving, by a transport robot, a request for transporting a servicerobot; automatically computing a location of the service robot;automatically moving the transport robot to the location of the servicerobot; automatically sending an a signal from the service robot to thetransport robot using a signal emitter incorporated into a mechanicalelement attached to the service robot; and automatically coupling, usingthe signal, the mechanical element to a carrier element attached to thetransport robot.
 2. The method of claim 1, further comprising:autonomously transporting the service robot by the transport robot to anew location; and automatically releasing the mechanical element fromthe carrier element by the transport robot.
 3. The method of claim 2,wherein the new location is automatically determined by: automaticallylocating a region of the new location; automatically moving of thetransport robot with the coupled service robot to the region;automatically receiving a sensor signal for locating the new locationwithin the region; and automatically moving of the transport robot tothe location using the sensor signal.
 4. The method of claim 2, whereinat least one of the location and the new location are work surfacesselected from the group consisting of: solar panels, ship hulls,airplane hulls, vehicle hulls, building windows, concentrated solarpower plant mirrors, and turbine blades.
 5. (canceled)
 6. The method ofclaim 4, further comprising orienting automatically the service robotparallel to at least one of the work surfaces to be serviced before theautomatic releasing, wherein the mechanical element comprises at leasttwo parts, and wherein the orienting is performed by articulationautomatically of the at least two parts.
 7. The method of claim 6,wherein the articulating is at least one of the group consisting of (i)an active articulation guided in real-time by controller instructions,(ii) fixed articulation for a preconfigured surface angle, and (iii) apassive articulation preconfigured for a range of surface angles,wherein the passive articulation comprises an articulating jointcomprising a fixed angle element and a flexible angle element.
 8. Themethod of claim 4, further comprising the transport robot servicing atleast one of the work surfaces.
 9. The method of claim 1, furthercomprising: automatically locating a region of the service robot by thetransport robot; and automatically moving of the transport robot to theregion of the service robot.
 10. The method of claim 1, wherein themechanical element is attached to the transport robot and the carrierelement is attached to the service robot.
 11. The method of claim 1,wherein the signal comprises an identification pattern.
 12. The methodof claim 1, further comprising transferring of at least one ofelectrical signals, power, and servicing materials between the transportrobot and the service robot.
 13. The method of claim 1, furthercomprising receiving automatically a drop-off request for the servicerobot, wherein the drop-off request comprises at least one of a locationand a region.
 14. (canceled)
 15. A robot coupling system comprising: atleast one service robot comprising a mechanical element, wherein themechanical element comprises a localization signal emitter; and at leastone transport robot configured to receive a request automatically fortransporting the at least one service robot, wherein each of the atleast one transport robot comprises: (a) a carrier element configured toautomatically couple with the mechanical element; (b) a sensorconfigured to automatically receive a localization signal from thelocalization signal emitter; (c) a motor configured to automaticallymove the at least one transport robot from a location to a new location;(d) at least one hardware processor; and (e) at least one storage unitcomprising processor instructions encoded thereon, the processorinstructions executable by said at least one hardware processor to causesaid automatic coupling, said automatic receipt, and said automaticmovement.
 16. The robot coupling system of claim 15, wherein theprocessor instructions are further executable by said at least onehardware processor to: receive a request for transporting the at leastone service robot; compute a location of the at least one service robot;move the at least one transport robot to the location of the at leastone service robot; receive a signal from the at least one service robotto the at least one transport robot using the emitter; mechanicallycouple the mechanical element to the carrier element using the signal;transport the at least one service robot by the at least one transportrobot to a new location; and release the mechanical element from thecarrier element by the at least one transport robot.
 17. The robotcoupling system of claim 15, further configured for: automaticallylocating a region of the at least one service robot by the at least onetransport robot; and automatically moving of the at least one transportrobot to the region of the at least one service robot.
 18. The robotcoupling system of claim 15, wherein the new location is determined by:automatically locating a region of the new location; automaticallymoving of the at least one transport robot with the coupled at least oneservice robot to the region; automatically receiving a sensor signal forlocating the new location within the region; and automatically moving ofthe at least one transport robot to the location using the sensorsignal.
 19. The robot coupling system of claim 15, wherein at least oneof the location and the new location are work surfaces selected from thegroup consisting of: solar panels, ship hulls, airplane hulls, vehiclehulls, building windows, concentrated solar power plant mirrors, andturbine blades.
 20. (canceled)
 21. The robot coupling system of claim19, further configured for: automatically orienting the service robotparallel to at least one of the work surfaces to be serviced before thereleasing.
 22. The robot coupling system of claim 15, wherein themechanical element is attached to the transport robot and the carrierelement is attached to the service robot.
 23. (canceled)
 24. The robotcoupling system of claim 15, wherein the mechanical element and carrierelement are adapted to transfer of at least one of electrical signals,power, and servicing materials between the transport robot and theservice robot. 25-30. (canceled)